EEVorlage

EE
Easy Extend News Changes Tutorial Downloads API Europython 07 presentation
Langlets
Gallery Coverage Macro P4D Py25Lite Teuton Zero
Tools
Console Test Trail
Misc
License Credits me The king of fiber-space What is it not
E a s y E x t e n d
Download API documentation Bytelets | Author: Kay Schluehr Date: 2008-08-16 Version: 1.2.3 |
1. Introduction 2. Using P4D 2.1 Install and run 2.2 Philosophers Football 2.3 Basic P4D element access 2.3.1 Attribute access using the @ operator 2.4 Declarative filters 2.4.1 Using wildcards in filters 2.4.2 Pitfalls 2.5 The elm keyword 2.6 namespaces 2.6.1 The p4d namespace 2.7 Python expressions in P4D elements 2.7.1 Object lists as element content 2.8 P4D comments 2.9 Element assignments and deletions ( new in P4D 1.2 ) 2.9.1 Assignments 2.9.2 Deletions 3. Support for XML languages 3.1. xmlstr and p4dstr 3.2 xml2p4d conversion from the command-line 3.3 The p4d namespace and its transformations 3.4 Translating XML comments and CDATA 3.4. Case Study - Adobe Flex 3.4.1 Import flexutils.p4d from Python 4. Bytelets ( new in P4D 1.2 ) 5. Token.ext & Grammar.ext 5.1 Token.ext 5.2 Grammar.ext 5.2.1 Translation of p4d_compound_stmt 6. History
# 1. Introduction A short version of an introduction to P4D can be summarized by a small formula: P4D = E4X without E and X but P and P4D trees A slightly more extended version goes like this: P4D ( = Python For Data ) is a full formed Python dialect that enables writing tree shaped data directly in source code. The main sources of inspiration for P4D were E4X which handles access to XML quite gracefully on language level and provides XML literals and SliP which preserves the structural properties of XML while being more apt to the concise Python syntax. As a result P4D keeps much of the original E4X design idea ( and also the syntax ) but drops XML literals and replaces them by more pythonic data structures. The transitions between P4D and XML are smooth. So you can use P4D to convert an XML document into a P4D datastructure edit and transform this structure and output an XML document again. P4D is implemented as an EasyExtend langlet which means among other things that P4D files compile to Python bytecode. So accessing P4D modules from Python is feasible ( it is actually feasible on source level as well but this will be explained later in this doc ) while accessing Python from P4D is obviously possible. # 2. Using P4D ## 2.1 Install and run P4D requires a Python 2.5 installation. Other versions of Python won't work with P4D. You can download P4D here. It is a Python package that is installed on the command line using the canonical python setup.py install <MyPythonDir>/lib/site-packages/EasyExtend/langlets/p4d P4D can be started as a Python script placed in the <MyPythonDir>/Scripts folder of you cd to the above directory and type python run_p4d.py A P4D specific console shall be opened immediately:
C:\lang\Python25\Lib\site-packages\EasyExtend\langlets\p4d>python run_p4d.py __________________________________________________________________________________ p4d On Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] __________________________________________________________________________________ p4d>
## 2.2 Philosophers Football A P4D statement is a tree of elements containing other P4D statements or elements. Following example encodes some data of Monty Pythons philosophers football game.
game = monty-python-football: location: "München" team(country="Deutschland"): player(no=1):"Leibniz" player(no=2):"I.Kant" player(no=3):"Hegel" player(no=4):"Schopenhauer" player(no=5):"Schelling" player(no=6):"Beckenbauer" # Beckenbauer obviously a bit of a surprise there player(no=7):"Jaspers" player(no=8):"Schlegel" player(no=9):"Wittgenstein" player(no=10):"Nietzsche" player(no=11):"Heidegger" team(country="Griechenland"): player(no=1):"Plato" player(no=2):"Epiktet" player(no=3):"Aristoteles" player(no=4):"Sophokles" player(no=5):"Empedokles" player(no=6):"Plotin" player(no=7):"Epikur" player(no=8):"Heraklit" player(no=9):"Demokrit" player(no=10):"Sokrates" player(no=11):"Archimedes" referees: main:"K'ung fu-tsze" assistants: linesman(no=1): "Saint Thomas Aquinas" linesman(no=2): "Saint Augustine" play: half-time(no = 1): team(country="Germany" goals = 0) team(country="Greece" goals = 0) half-time(no = 2): team(country="Germany" goals = 0): substitutes: subst(p4d:in="Marx" p4d:out="Wittgenstein") matchwinner: "False" team(country="Greece", goals = 2): substitutes goal-scorer(goal=1): "Sokrates" goal-scorer(goal=2): "Sokrates" matchwinner: "True"
## 2.3 Basic P4D element access Now we show how to access elements using P4Ds syntax. The game object which is an ordinary Python object serves as an entry point.
p4d> game P4D<monty-python-football>
Notice here one particular detail. In P4D one can write hyphenated names: NAME ('-' NAME)*. However one can write them within P4D elements only. In other contexts the same syntactical form will be interpreted as an arithmetic difference.
p4d> game.location [P4D<location>]
We can access elements using the dot-operator. The return value is a P4DList object. A P4DList is derived from Pythons list type. It provides some additional methods which it shares with basic P4D objects. So one can access text directly for instance:
p4d> location = game.location p4d> location.text()
This however works only if the P4DList contains exactly one element. Otherwise an AttributeError is raised as in the following example.
p4d> game.team.player Traceback ... AttributeError...
Notice that the attributes in the attribute list as in
team(country="Greece" goals = 2)
don't have to be separated by comma. Commas are optional though and you can write the element in Python style:
team(country="Greece", goals = 2)
### 2.3.1 Attribute access using the @ operator We can access P4D object attributes either using the attributes() method or we use the @-operator which is a syntactical shortcut.
p4d> game.team[0].@country "Deutschland" p4d> winning_team_goals = int(game.play.child("final-score").team[1].@goals)
In the latter case we have to use the child() method of the P4D element to access the final-score element. This is necessary because we do not use final-score within the context of an P4D object definition. Another use of the @-operator is to select all attributes of an element. This is done by the combination of @ with the wildcard symbol *.
p4d> game.play.child("half-time").team[1].@* {"country":"Greece", "goals": "2"}
## 2.4 Declarative filters Subelement filters are an E4X feature also available in P4D. A subelement filter is basically a statement about element properties used to filter subelements of the element to which the filter is applied.
X.(filter-declaration) --> for item in X.children(): filter-function(item)
The main purpose of filters is navigation. Intuitively speaking one tries to reduce a list of child nodes to a list of exactly 1 element. On this 1 element list one can apply another filter and so on. Example 1:
a = a: b(x = 0 y = 1): "abc" b(x = 0 y = 2): "a" c(x = 1 y = 2): "abc" c(x = 0 y = 2): d: "def" d: "efg"
With on element a we can apply several different filters:
p4d> a.(@x == 0) # -- elements X in a with X.@x == '0' [P4D<b>, P4D<b>, P4D<c>] p4d> a.b.(.text() == "abc") # -- elements b in a.b with b.text() == "abc" [P4D<b>] p4d> a.c.(d == "abc") # -- elements c in a.c with subelement d and d.text() == "efg" [P4D<c>] p4d> a.(@x == 1 or @y == 1) # -- elements X in a with X.@x == 1 or X.@y == 1 [P4D<b>, P4D<c>] p4d> a.b.(.children() == []) # -- elements b in a.b with b.children() == [] [P4D<b>, P4D<b>]
Example 2: Applying filters to the philosophers football game:
p4d> game.(@country == "Deutschland").player.(@no == 3).text() 'Hegel' p4d> game.play.children("half-time")[1].team.(matchwinner == "True").@country 'Greece'
As you see the "half-time" element must be selected using the children() object method because "half-time" is not a Python name. ### 2.4.1 Using wildcards in filters In the second example above we mentioned the subelement tag explicitely in the filter declaration:
p4d> a.(b == "abc") # lambda this: this.text( ) == "abc" and this.tag == 'b' [P4D<b>]
Using a single underscore _ instead of b leads to a different translation where the clause this.tag == '_' will be just omitted:
p4d> a.(_ == "abc") # lambda this: this.text( ) == "abc" [P4D<b>, P4D<c>] p4d> a.(_.text()[0] == "a") # lambda this: this.text( )[ 0 ] == "a" [P4D<b>, P4D<b>, P4D<c>]
### 2.4.2 Pitfalls Filters operate just skindeep i.e. on the level of child elements of a current node. You see this when trying to use a filter declaration to filter among nodes on a deeper level of hierarchy e.g.
a.(c.d.text() == "def") # lambda this: this.d.text() == "def" and this.tag == 'c'
Looks quite o.k. but isn't. Actually this query fails with following exception: P4DAccessError: Cannot access P4D text on P4DList with length!=1. The message means that c.d is a list of nodes of length != 1 ( two nodes in our example ) and P4D only treats P4DList objects of length = 1 like P4DNode objects! ## 2.5 The elm keyword So far we have assigned P4D elements to regular Python names using the assignment form py_name '=' p4d-element However each P4D element starts itself with a name so we have py_name '=' p4d_name ... Whenever a p4d_name is also a valid py_name valid we can reduce boilerplate and just write elm p4d_name ...
# explicit assignment: philosopher = philosopher: name: "Gottfried Wilhelm Leibniz" birth: date: "1646-07-1" city: "Leipzig" death: date: "1716-11-14" city: "Hannover" # implicit assignment: elm philosopher: name: "Georg Wilhelm Friedrich Hegel" birth: date: "1770-08-27" city: "Stuttgart" death: date: "1831-11-14" city: "Berlin"
Definitions using elm will always be expanded into the explicit assignment form. Implicit assignment cannot be used with hyphened names such as monty-python-football or prefixed names like p4d:if. In both cases a SyntaxError is raised and an explicit assignment using a valid Python name is recommended. Why elm and not the more intuitively appealing elem? An elem keyword would be like var or val. Any second Python program has defined one and would be invalid. ## 2.6 namespaces XML and P4D elements provide context and scope. Elements shall therefore be sufficient for most practical disambiguation purposes. However XML moves beyond this and provides XML namespaces which lets you mix elements of different XML based domain specific languages freely. Although P4D supported XML namespaces initially just for compliency reasons A namespace consists of two parts: a prefix and an URI ( or URN ). The URI shall serve as a strong name providing real means of contextual disambiguation. The prefix is kind of a variable that binds to an URI. When in use namespace prefixes are separated by colon from a so called local name. Prefxixes are the visible clue you have most likely noticed in XML elements like this <xsl:stylesheet version "1.0" ...> Library authors who take namespaces really serious substitute prefixes by their corresponding URIs ( URNs ) entirely when XML documents are getting dumped. The elementtree package of the Python standard library is an example of this practice. P4D considers namespaces prefixes basically as an additional syntactical detail. One can retrieve namespace information from elements though. This doesn't mean much more than a variable lookup.
a = a: b:c(xmlns:b = "http://bla.bla.bla") p4d> a.b::c P4D<b:c> p4d> a.b::c.namespace() <EasyExtend.langlets.p4d.p4dbase.P4DNamespace object at 0x01487C10> p4d> b_ns = a.b::c.namespace() p4d> b_ns.prefix 'b' p4d> b_ns.uri 'http://bla.bla.bla'
The doublecolon operator :: is used to express full element access of a prefixed name. Alternatively one can use the child method with the full tag name:
p4d> a.child("b:c") P4D<b:c>
If an element is not prefixed the invocation of namespace() on it returns a P4DNamespace object with empty content.
p4d> a.namespace().uri # None p4d> a.namespace().prefix # None
If an element is not prefixed the invocation of namespace() on it returns a P4DNamespace object with empty content. ### 2.6.1 The p4d namespace There is one default namespace URL in P4D which is http://fiber-space.de/p4dns. The corresponding prefix is p4d. The p4d namespace is introduced for a very particular reason: it wraps language keywords. Just look at the following example:
if: while(x=1): pass
P4D would fail to parse this expression. Keywords are treated special by the parser. However you might prefix these element tags using p4d:
p4d:if: p4d:while(x=1): p4d:pass
In this case the parser reads p4d:NAME as a single token. How to navigate through this structure? For namespace information retrieval you can simply access elements using a double colon:
a = p4d:if: p4d:while(x=1): p4d:pass p4d> a.p4d::while.@x '1'
Alternatively one can use the child(..) accessor method
p4d = p4d:if: p4d:while(x=1): p4d:pass p4d> a.child("p4d:while").@x '1'
## 2.7 Python expressions in P4D elements So far we considered just static content. Content of P4D attributes or elements had to be strings or numbers. If the content was a number it was automatically converted into a string. Now we want to place arbitrary Python expressions within P4D elements or attributes. The value of the expression shall be either a string or a P4DNode object. Embedding however might cause trouble. Just look at the following example:
p4d> x = 1 p4d> y:x P4D<y:x>
Here y is understood as a namespace prefix and the assignment of x is ignored. In order to let the value of x be the content of y we write:
p4d> x = 1 p4d> y:&x P4D<y>
The ampersand operator can prefix an arbitrary Python expression. So the following more compex expression shall work
p4d> x = 0 p4d> y:&(x if x>0 else -x) P4D<y>
The expression is evaluated when the root P4D element is created. I considered lazy evaluation ( call by need ) as well. This enables dataflow and partial constructions. However it is not available in this release. Attribute values actually wouldn't require the ampersand operator because there is nothing to disambiguate. However I abandoned modality in favor for simplicity: evaluation in P4D elements always only takes place when they are prefixed by an ampersand operator. Notice that omitting the ampersand operator leads to failure in case of general expressions:
p4d> x = 0 p4d> y:(x if x>0 else -x) # wrong - expr needs to be prefixed with '&' -> ParseError p4d> y(a = x if x>0 else -x): 0 # wrong - expr needs to be prefixed with '&' -> SyntaxError p4d> y(a = x): 0 # wrong - expr needs to be prefixed with '&' -> SyntaxError p4d> elm y(a = "x if x>0 else -x"): 0 # o.k. strings are permitted p4d> elm y(a = 42): 0 # o.k. numbers are permitted p4d> elm y(a = &x if x>0 else -x): 0 # o.k. ampersand prefix used p4d> y.@a '0'
### 2.7.1 Object lists as element content This feature enables a more compact notion of lists of elements. It is not directly representable in XML and might be the single biggest departure. Suppose you have a list L = [1,2,3,4] and for each entry in L you want to create an Item element with the entry as content. The usual way to do this is to write:
# variant 1 elm X: Item: 1 Item: 2 Item: 3 Item: 4 # variant 2 using the ampersand operator elm X: Item: &L[0] Item: &L[1] Item: &L[2] Item: &L[3]
In P4D this boilerplate can be eliminated entirely and you can simply write
# variant 1 elm X: Item: [1,2,3,4] # variant 2 using the ampersand operator elm X: Item: &L
Those lists will be expanded and for each entry an Item object is created internally:
p4d> X.Item [P4D<Item>, P4D<Item>, P4D<Item>, P4D<Item>]
Attributes will be replicated and P4D nodes become embedded:
elm a: b(x = 0): [1,2,3] elm Y: Item: &a.b p4d> Y.Item [P4D<Item>, P4D<Item>, P4D<Item>] p4d> Y.Item[1].b.text() '2' p4d> Y.Item[1].@x '0'
### 2.7.2 Non P4D element content A common use case of the ampersand operator is to assemble a P4D object from smaller parts:
elm A: a(x = 0): [1,2] elm B: a(x = 1): [3,4] elm C: &A &B
We can access A and B from C immediately. It is less obvious what happens when A and B are not references to P4D objects. Suppose you have the following statement:
x = "Some Text" y = False elm A: B: &x &y
What can be used as content here? Shall an exception be raised because x and y can be both content? Shall only y be content because it is a string? Shall y be used because it is the last definition in the row or x because it is the first one? Actually I decided to use both ones and defined a new type which is called P4DContentList. P4DContentList is derived from list and you usually don't have to access it explicitely except for the special case of having more than one objects that can be content. The implementation of P4DContentList is rather simple:
class P4DContentList(list): def __str__(self): return ''.join(str(item) for item in self)
The P4DContentList object will be printed as an ordinary string not as a list. Notice that the sequence of non P4D objects is not preserved. ### 2.7.2 None is not content The None object will not be inserted as content. Writing a P4D element in this style
y = None elm A: &None &y
yields an element A without children and without content. This can be used to optionally insert P4D elements in A:
y = (compute_element(args) if cond else None) elm A: &y
## 2.8 P4D comments Using single or triple quoted strings are used to denotate P4D content within P4D elements. They can't be used to create XML like comments which are significant. Signifcant means that the comments are usually not dropped by the parser but content is sourced out into comments that doesn't fit well into the document structure otherwise. The comment style of P4D is {* ... *}.
elm foo: {* This is a comment *} """... and this is content. """
You can access a comment by either using the comment() object method or the child(tag) object method with tag = '*'. Notice that a P4D comment is a regular P4D element! A P4D comment is much like a docstring in Python which is just a usual string. ## 2.9 Element assignments and deletions ( new in P4D 1.2 ) ### 2.9.1 Assignments You can modify content or replacing elements by assigning values to them:
elm a: b: 1 elm b: c: 2 p4d> a.b.content() '1' p4d> a.b = 2 p4d> a.b.content() '2' p4d> a.b = b p4d> a.b.c.content() '2'
Notice that you can't assign a P4D element A to a P4D element B. When you really want to change a tag you have to rebind the tag attribute:
p4d> a.b[0].tag = 'x' p4d> a.x [P4D<x>]
Further limitations: you can't assign to b if b is a list of nodes with length > 1. ### 2.9.2 Deletions Subelement deletions are as easy as assignments
p4d> elm a: .... b .... c .... p4d> a.children() [P4D<b>, P4D<c>] p4d> del a['c'] # remove 'key' p4d> a.children() [P4D<b>]
A KeyError is raised when no child element with tag = 'c' is available. # 3. Support for XML languages Support for XML languages is an obvious goal of P4D. The transition between P4D and XML shall be smooth since internally the same datastructure is used to hold elements. In section 5.1 we start with some conversion groundwork. Section 5.2. examines Adobe Flex and MXML as one particular use case. ## 3.1. xmlstr and p4dstr
p4d> elm philosopher: .... name: "Hegel" .... main-work: "Phaenomenologie des Geistes" .... first-edition: 1807 .... p4d> hegel_data = philosopher.xmlstr() p4d> print hegel_data <philosopher> <name>Hegel</name> <main-work>Phaenomenologie des Geistes</main-work> <first-edition>1807</first-edition> </philosopher> p4d>
You can reconstruct the original P4D element with the same ease:
p4d> philosopher = P4D.from_xml(hegel_data) p4d> print philosopher.p4dstr() philosopher: name: "Hegel" main-work: "Phaenomenologie des Geistes" first-edition: "1807"
You can print additional data that are optional e.g. a standard XML declaration:
p4d> print philosopher.xmlstr(xml_declaration = True) <?xml version="1.0" encoding="UTF-8" ?> <philosopher> <name>Hegel</name> <main-work>Phaenomenologie des Geistes</main-work> <first-edition>1807</first-edition> </philosopher>
Another option is to create an assignment when a P4D representation is created:
p4d> print philosopher.p4dstr(name = "p4ddoc") p4ddoc = philosopher: name: "Hegel" main-work: "Phaenomenologie des Geistes" first-edition: "1807"
## 3.2 xml2p4d conversion from the command-line Let's say you have an XML file foo.xml and you want to convert it into P4D. Then you can use the --xml2p4d command line option and run python run_p4d.py --xml2p4d foo.xml This will create a file foo.p4d in the directory of foo.xml. If you omit the name foo but just write python run_p4d.py --xml2p4d *.xml all files with extension xml in the directory you are pointing to will be converted to p4d. ## 3.3 The p4d namespace and its transformations The namespace that is characterized by the URL http://fiber-space.de/p4dns and the prefix p4d is somewhat special as we have noted in section two already. Now take a look at the following XML element kept from a catalog.xml file which lists resources of Adobe Flex components.
<library path="library.swf"> <script name="mx/core/mx_internal" mod="1146526998000" > <def id="mx.core:mx_internal" /> <dep id="AS3" type="n" /> </script> </library>
It would translate into
library(path="library.swf"): script(name="mx/core/mx_internal", mod="1146526998000"): def(id="mx.core:mx_internal") dep(id="AS3" type="n")
and this fragment would immediately fail to compile because def which is used as an element name is a P4D ( and Python ) keyword. That's why the p4d namespaces prefix is added automatically by default:
p4d> p4d_lib = P4D.from_xml(xml_lib).p4dstr() p4d> print p4d_lib library( path="library.swf" ): script( name="mx/core/mx_internal", mod="1146526998000" ): p4d:def( id="mx.core:mx_internal" ) dep( type="n", id="AS3" )
The reverse operation works also in the other direction i.e. when creating the XML fragment from P4D. Applying xmlstr()
p4d> print P4D.eval(p4d_lib).xmlstr() <library path="library.swf"> <script name="mx/core/mx_internal" mod="1146526998000" > <def id="mx.core:mx_internal" /> <dep id="AS3" type="n" /> </script> </library>
We can also suppress adding the p4d prefix on loading from XML or we can suppress stripping the prefix away:
p4d> print library.xmlstr(strip_p4d = False) <library path="library.swf"> <script name="mx/core/mx_internal" mod="1146526998000"> <p4d:def xmlns:p4d="http://fiber-space.de/p4dns" id="mx.core:mx_internal"/> <dep type="n" id="AS3"/> </script> </library>
Not only has the namespace prefix been preserved but also the correct namespace declaration has been inserted. The latter will be done only once. ## 3.4 Translating XML comments and CDATA XML comments will be translated into P4D comments. That's why P4D comments were introduced in the first place. A slightly different translation will be performed for CDATA sections. A P4D comment is fully specified by {* content *}. If we replace the single star by a double star the P4D comment will be translated into <![CDATA[ content ]]>.
p4d> elm stuff: .... ''' .... Some text .... ''' .... {* and a simple comment ... *} .... {** and a cdata .... section .... **} .... p4d> print stuff.xmlstr() <stuff> Some text <!-- and a simple comment ... --> <![CDATA[ and a cdata section ]]> </stuff> p4d> p4d> print P4D.from_xml(stuff.xmlstr()).p4dstr() # and the other way round stuff: "Some text" {* and a simple comment ... *} {** and a cdata section **} p4d>
Whitespace handling is not perfect. I hope I'll drive it perfection in the future :) Notice that CDATA comments are treated much like regular P4D comments i.e. they are normal P4D elements, there is a CDATA() method used to access a CDATA section and there is an element tag which is '**'. There is obviously some redundancy about comments in P4D but usually you don't have to care much about them unless you are converting from / to XML for particular reasons. ## 3.4. Case Study - Adobe Flex Adobe Flex is a framework used to support so called Rich Internet Applications ( RIA ). This is such a hot topic right now that it is somewhat too hot for me to discuss here. So my interest is focussed mainly on MXML which is an XML language for declarative GUIs descriptions and part of Flex. The idea of using XML to describe programming language constructs has been critizised a lot in the past and I mostly agree with it myself. However one has to change the perspective to do justice about it. When you consider MXML from the point of view of a tool vendor who offers a GUI designer that enables roundtrip engineering in a safe way, it makes good sense. MXML becomes essentially a human readable serialization format for the FlexBuilder tool. This is much in alignment with Flash which is also basically a tool and a player. However Adobe has released an SDK a while ago and now you can build your applications from the ground up in a way that is more appealing to programmers like me ( I don't alway rate flexibility over comfort even when programming but I nevertheless tend to be fundamentalist as a programmer ). I added the flexutils.p4d module to P4D. It might not be for everyone and I'll refuse to explain Flex related issues ( like configuration ) here. If interested read the code and adapt it to your advantage. It is not much anyway. However flexutils.p4d touches some framework related topics which might be of interest for a general readership. ### 3.4.1 Import flexutils.p4d from Python Any langlet ( *.p4d ) module compiles to Python bytecodes and can therefore be imported as a bytecode ( *.p4d ) file. But now suppose you have a Python module that imports a p4d file. That's what run_p4d.py does:
# run_p4d.py # ---------- import flexutils # imports a p4d-module
and you change flexutils.p4d. Only as long as flexutils.p4d gets compiled before run_p4d.py is executed everything works well and the most recent changes will be notified. Otherwise the old flexutils.pyc will be imported. In order to enforce recompilation on change we replace the import directive:
# run_p4d.py # ---------- import EasyExtend.eeimporter as eeimporter import langlet as p4d_langlet flexutils = eeimporter.import_from_langlet(p4d_langlet, "flexutils")
The import_from_langlet function is generic over langlets. You need to pass the right langlet of course to apply the correct transformations on flexutils.p4d. This import pattern doesn't have to be applied when you import Python modules from P4D.
| You can't run if __name__ == '__main__': BODY defined inside a .py module from P4D. You can run this inside .p4d modules though. The reason is that EasyExtend transforms this statement into an ordinary method call and calls this method but only for langlet modules not for Python modules that won't get transformed at all.
  # 4. Bytelets ( new in P4D 1.2 ) A detailed treatment of Bytelets can be found here. At this place it shall only be mentioned that Bytelets affect P4D as a whole in following respects: Two more namespaces are introduced. Their prefixes are bl and bl-schema. Hexadecimal literals like 0x89 or 0xA57 are not mapped on integers in P4D but on Hex objects. Moreover it is possible to write sequences of such numbers separated by a space e.g. 0x01 0x78 0x67. This is an important departure from Python and might break existing semantics. # 5. Token.ext & Grammar.ext The syntactical extensions involve both the lexer and the parser. ## 5.1 Token.ext
  DOUBLECOLON OPERATOR NAME P4_NS Name P4D_Comment | ::= ::= ::= ::= ::= ::= | '::' OPERATOR_DEF | DOUBLECOLON Name ('-' NAME)* | P4_NS 'p4d' ':' NAME A_CHAR ( A_CHAR | A_DIGIT )* '{' '*' ('*'|ANY)* '*' '}'
Most noteworthy is the redefinition of NAME and the introduction of Name that carries on the standard definition of the Token module. Enabling hyphenated names of the kind monty-python-football is problematic in other contexts where it has to be understood as an arithmetic difference or isn't permitted entirely. The transformation actually takes care:
p4d> a = b = 0 p4d> a-b 0 p4d> a-b:0 P4D<a-b> p4d> import a-b Traceback... ... SyntaxError: invalid syntax. No hyphenation permitted in Python names.
With P4_NS we can escape keywords. Just look at the difference in behaviour:
p4d> a:pass:"" Traceback... ... ParserError: Failed to parse input 'pass' at (line 1 , column 6). a:pass:"" ^^^^ p4d> p4d:pass:"" P4D<p4d:pass>
  This wouldn't be feasible with the current parser if we won't deal with "p4d:pass" as a single chunk and doing some postprocessing. ## 5.2 Grammar.ext
  compound_stmt p4d_compound_stmt p4d_element p4d_attribute_list p4d_attribute p4d_suite p4d_simple_stmt p4d_stmt p4d_expr p4d_accessor trailer p4d_name p4d_attr_access atom HEXNUM subscript | ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= ::= | if_stmt | while_stmt | for_stmt | try_stmt | ... | p4d_compound_stmt ['elm' | NAME '='] p4d_element ':' p4d_suite p4d_name ['(' [p4d_attribute_list] ')'] p4d_attribute ([','] p4d_attribute)* p4d_name '=' ['&'] test p4d_simple_stmt | NEWLINE INDENT p4d_stmt+ DEDENT (p4d_element | p4d_expr) NEWLINE p4d_simple_stmt | p4d_compound_stmt '&' '.'* test | '(' [ p4d_expr (',' p4d_expr)*] ')' | '[' [ p4d_expr (',' p4d_expr)*] ']' | STRING | NUMBER | HEXNUM | P4D_Comment '.' p4d_attr_access |'::' NAME | '.' '(' test ')' '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME | p4d_accessor NAME [':' NAME] '@' ( NAME | '*') atom: ('(' [yield_expr|testlist_gexp] ')' | '[' [listmaker] ']' | '{' [dictmaker] '}' | '`' testlist1 '`' | NAME | NUMBER | STRING+ | HEXNUM | p4d_attr_accesss Hexnumber+ '.' '.' '.' | test | [test] ':' test [sliceop] | [test] '::' test | '::' test | [test] ':'
Rules that substitute those of Pythons grammar are atom, trailer and compound_stmt. P4D specific statements are realized as compound statements and they follow the rules of Pythons indentation. ### 5.2.1 Translation of p4d_compound_stmt A P4D compound statement is translated into a P4D object constructor call. Instead of giving a formal specification here we simply show some translations of common P4D compound statement structures.
p4d> a: .... b:0 .... [python-source> P4D( mapeval(['a', {}, [['b', {}, ['0']]]] , globals( ))) <python-source] P4D<a>
The list that is passed to mapeval has the canonical form [TAG, ATTRS, CHILDREN, TEXT] The textual content isn't stored separately but among the CHILDREN. P4D automatically stringifies numbers such as the 0. Actually the content can be omitted completely
p4d> a: .... b .... [python-source> P4D( mapeval(['a', {}, [['b', {}, []]]] , globals( ))) <python-source] P4D<a>
Here is another example using attributes ( and also namespaces ).
p4d> p4d:if: .... p4d:while(x=1 y=2): .... p4d:pass .... [python-source> P4D( mapeval(['p4d:if', {}, [['p4d:while', {'y': 'evaltostr_1', 'x': 'evaltostr_2'}, [['p4d:pass', {}, []]]]]] , globals( ))) <python-source] P4D<p4d:if>
What's more remarkable here are the attribute value strings 'evaltostr_1' and 'evaltostr_2'. These strings are the reason for the presence of mapeval. The mapeval function seeks for strings beginning with "evaltostr_" and "evaltoobj_" and evaluates the appended part of the string which is "1" and "2" here. Finally it turns them into strings again. What is redundant in our particular example makes sense in the more general case.
p4d> a = 89 p4d> b(x = 1+a):"" [python-source> P4D( mapeval(['b', {'x': 'evaltostr_1 + a'}, []] , globals( ))) <python-source] P4D<b>
Now we can access the evaluated attribute value:
p4d> c = b(x = 1+a):"" p4d> c.@x '90'
# 6. History
Date | Version | Change
2008-08-16 | 1.2.3 | Bugfix in evalutils: elements of the kind A: B: ["x1", "x2"] C: ["y1", "y2"] caused problems when C was expanded. Add P4DContentList for storing content that stems from multiple inputs as in the following case: A: "A string" "Another String" add get() method to P4D and P4DList objects. It's close to child() but unlike child it doesn't raise a ValueError exception if child element is unavailable.
2008-08-13 | 1.2.2 | Bugfix in xmlutils: attribute accepts empty string values. Bugfix in p4dutils: xml2p4d conversion failed due to import error on P4D when not started from P4D.
2008-08-10 | 1.2.1 | Worst case error i.e. grammar bug fixed. Token.ext specified (* ... *) as comments within P4D elements. I completely overlooked that this can't be used since foo(*bar) is a valid. This bug remained unrecognized because Grammar / Token are separated from each other. We need to replace (* ... *) by {* ... *}. This is both Python 2.X as well as Python 3.0 compliant. This change is not downwards compatible!
2008-07-12 | 1.2 | One can now set attribute values and content easily without referring to the internal datastructure. Introduction of Bytelets and the bl namespace which is supported on language level. Adding a Hex object in this context. elm foo:bar: ... works now like an assignment bar = foo:bar:... which is valid for all for all non keywords and non hyphened names.
2008-07-11 | 1.1.1 | Fix: remove py4as3 import from xmlutils and p4dbase
2008-07-11 | 1.1 | Fix XML formatAttributes() in xmlutils. Make locals() available for mapeval. Add flexutils module Add elm keyword Add P4D comments Fix double-colon access for namespaces with p4d prefix. Add element lists or tuples. Add P4D.eval() Add CDATA handling Change filter semantics
2008-06-26 | 1.0 | Initial release